It is known that photographs or images can have information contained within the image area (for example: bar codes, date and time). It is also known that digital information (computer readable) can be applied to the image area. There are have been several endeavors to embed digital data in images, for example for copyright protection. For various reasons as noted below, none of these methods are appropriate for embedding data in images containing general photographic content with minimal visibility while retaining the existing quality of the image. It is also a desirable feature to be able to encode the data in the image either optically or digitally. The techniques that do address embedding data in images are limited to special subsets of images including uniform fields of color, graphically generated images with large areas of uniform color (such as the pie-charts of business graphics), or text. Known prior art methods of embedding digital information in images have several drawbacks for application to images having photographic pictorial content. With these technologies, the image may be partially or completely distorted, the embedded data can be digitally removed, or the method may require a significant modification to hardware in order to implement the technology. Some of these technologies use areas outside of the image area (glyphs or bar codes) because the techniques are largely destructive of the image itself and require a uniform background to read the information. Some methods create visible distortions within the image that can be unsightly and/or undesirable. Other techniques such as applying logos, labels, or stickers to an image can be easily hidden (covered) and can also be unsightly. Magnetic strips applied to the image require the scanning devices to be specially modified with a magnetic detector to allow the strip to be sensed. Most of these methods provide a binary solution (that is, they allow only a yes/no or go/no-go solution).
U.S. Pat. No. 4,939,515 issued July 1990 to Adelson shows a technique for embedding digital information in television signals. The technique first decomposes the image into a resolution pyramid, the layer of the pyramid having the highest spatial frequencies is omitted and a Hadamard-based bit stream is substituted. This approach relies on the fact that the optical spot of a CRT contributes far more strongly to the high frequency attenuation of the CRT MTF than any of the transmission components. Therefore, the signal can be encoded, transmitted, and decoded by the electronics, while the optics of the CRT attenuates the high frequencies such that the embedded digital stream is not visibly objectionable. It also relies on the property of human visual psychophysics where the high spatial frequency sensitivity of the visual system is limited to low temporal frequencies, and thus by having the high frequency digital stream change from TV field to TV field, the human eye is not very sensitive to it. This method can achieve data embedding that is entirely invisible under some circumstances (i.e., on TV monitors with poor MTF characteristics, far viewing distances, or certain image sequences). This approach also requires the encoding step to be performed digitally. Another shortcoming is that the technique relies on systems with low physical MTF relative to the nyquist frequency, which is generally not true across the widely varying image qualities of current imaging systems. Yet another shortcoming is that it relies on the high frequency digital signal information having a high temporal frequency. This is impossible to achieve in still images in hard-copy form.
U.S. Pat. No. 4,972,471 issued November 1990 to Gross uses notch filters and temporal modulation of their output in order to embed digital information inaudibly in recordings. One of the stated applications is toward the monitoring of broadcast recordings for copyrights. The recorded signal is embedded in a two pass algorithm. First the recording is scanned and decomposed into several notch frequencies (frequencies corresponding to the musical diatonic scale are suggested) to look for the best placement of a start code. This is decided by use of the properties of audio masking known from the science of acoustic psychophysics, and the method looks for the presence of two neighboring frequencies, one having a sufficiently higher amplitude than the other. Audio masking dictates that the lower amplitude frequency can be removed without the human observer being able to hear the difference. Once this temporal location is decided in the first pass, the second pass of the algorithm determines the temporal starting point of the code and the notch frequencies used and calibrates this against the existing parallel SMPTE code. This data is stored by those wishing to test identity of the signal (i.e., the owners), and is not made available to those wishing to corrupt the embedded data (i.e., the copyright infringer). The second pass actually embeds the start code pattern and the identifying information in a specified temporal gap after the appearance of the start code. The bits of the identifying information are encoded as temporal modulation of the notch filter content. Although the patent suggests that the method may be applied to images, the drawbacks in trying to apply this approach to images include:
1. there is no equivalent of the SMPTE time code in images and therefore finding an equivalent of the start code would be very difficult; PA1 2. the placement of the embedded information depends on the content of the recorded signal in order to determine where (both in frequency and in time) to place the embedded information, which requires a digital or complex analog system; and PA1 3. the data is not spatially spread throughout the image. Consequently, if the start code or the identifying code is accidentally blocked by unintentional or intentional corruptions of the signal, which in the optical imaging application would include such defects as dust and scratches, the data will be lost.
U.S. Pat. No.: 5,091,966 issued February 1992 to Bloomberg; U.S. Pat. No. 5,128,525 issued July 1992 to Stearns, et al.; U.S. Pat. No. 5,168,147 issued December 1992 to Bloomberg; U.S. Pat. No. 5,221,833 issued June 1993 to Hecht; and U.S. Pat. No. 5,245,165 issued September 1993 to Zhang, comprise a family of patents that address embedding digital information invisibly within in hard-copy images. However, they are limited to images containing significant uniform areas of color, such as found in computer-generated graphics for business and science data presentation, or images containing text. The approach they have taken is to use small localized and non-overlapping gray-scale shapes to carry the bit information. The shapes are referred to as glyphs, and are small enough and closely packed so that they visually merge into a uniform field when printed on a sufficiently high resolution device (300 dpi). This technique works in electrophotographic copying machines because such machines have the ability to capture very high resolution edges (the amplitude dimension is often captured with low resolution, however) in order to keep the edges of the text sharp in appearance. The various patents address different shapes, different methods of dealing with scale changes, and different methods of dealing with image skew, such as rotation. The technique is basically a matched filter method used to directly search the embedded image for the glyphs. The technique does not work with images consisting of photographic subject matter including such image features as texture and grain noise because the texture and grain noise would mask the detection of the glyphs.
Several articles also describe various methods of embedding data for copyright protection. Schyndell, "A digital Watermark" IEEE ICIP-94 (1994) presents a method based on modulation of the least significant bit (LSB) of a digital image. This method requires digital encoding and the message would be lost if that bit is truncated, or if the noise levels in an image are increased, which is likely in the scanning of the digital image out to hard copy form, as well as scanning the image in from hard copy. Sapwater et al., "Electronic Copyright Protection", Photo Electronics Imaging, Vol 7, No. 6, pages 16-21 (1994) explores the issues in copyright protection of digital images, but does not present a satisfactory solution. The proposed solution is to digitally place a copyright symbol in the Yellow layer of a Cyan, Magenta, Yellow, and Black (CMYK) version of a color image. When the image is shown in color, the symbol is hard to see because of the weak blue-yellow sensitivity of the human eye. However, when the color image is broken down into C,M,Y, K layers, and the Y layer is displayed as a black and white image, it is possible for the human observer to see the copyright symbol. This technique has the disadvantages that it requires the encoding to be performed digitally, and further requires human intervention to place the symbol in an area where it is not likely to degrade the image (such as avoiding the subject's face). It also has the disadvantage that the detection of the copyright is not automated. Bender et al, "Techniques for Data Hiding", SPIE Proceedings 2420 (1995) present two techniques for embedding small amounts of digital data in images. One of these is termed Texture Block Coding and involves digitally copying a specifically shaped region of texture from one portion of the image to another. The bit pattern is encoded in the shape. Two disadvantages of this technique are that the encoding must be done digitally, and further it requires a skilled image operator to select similarly textured regions to perform the swap. Further, the Bender paper does not explicitly state how to code digital information in the shapes, and has not performed such studies. Another method proposed by Bender is called Patchwork and involves slight offsets of pixels (which may also be low-pass filtered blob-like regions consisting of large numbers of actual pixels) in positive and negative directions so that as a specific path is taken through the image, the expected value of the differences deviates strongly from the mean of the image. This approach is intended to code only one bit of information, but it may be extended to a small number of bits (&lt;8) by methods not explicitly defined in the paper. No way of implementing the technique optically or with hard copy was disclosed. A problem with applying the Patchwork technique to hardcopy is that of finding the proper pathway through an image after it has been converted to a digital image by scanning due to size and rotation issues. Walton, "Image Authentication for a Slippery New Age", Dr. Dobbs Journal, April, page 18-26 (1995) presents a technique for data embedding for image authentication using a checksum method. The technique is easily corrupted and would not likely survive a hardcopy form. The Walton technique has the further disadvantage that it requires the encoding to be performed digitally.
There is a need therefore for a technique for embedding digital data in images that can be implemented either digitally or optically, that will not visibly distort the image, is not easily corrupted by image content or defects, and is not lost when the image is cropped, rotated, or resized.